The present invention is in the field of data-packet-routing over data-packet-networks, in particularly the hardware and software comprising a data packet router. The invention pertains particularly to methods and apparatus for upgrading software running on a data router while the router remains online and operational.
At the time of the present patent application demand for increased data capacity and efficiency in Internet traffic continues to increase dramatically as more individuals and businesses increase their use of the Internet. The ever-increasing demand also drives development of equipment for use on the Internet such as data packet routers. A number of enterprises are now developing routers that are capable of faster and higher-capacity handling of data packets routed over the Internet.
The Internet, operating globally, comprises components from a wide variety of manufacturers. It is, of course, necessary that such equipment conform to certain hardware, software and connection standards, and operate by certain data transfer standards and protocols. These standards are all well-known to the skilled artisan.
As new and better equipment for routing data traffic in the Internet continues to be developed, researchers developing such equipment are endeavoring to include such as fault tolerance, diagnostic capabilities, and redundancy into the equipment, links between instances of routing equipment, and in implementation of routing protocols.
One of the important developments in Internet technology as of the time of filing of the present patent application is in development of faster and scalable routers utilizing a distributive-processor concept. The present inventors, for example, are involved in development of what are known in the art as Terabit routers, capable of much higher packet transfer rates than for routers currently available in the art. These routers are scalable to higher and higher overall data-transfer rates and capacity, by allowing upwards of seven thousand interfaces to network traffic.
An example of a Terabit router as known to the inventor comprises three types of processor-equipped cards adapted for controlling and routing data. Each card has it""s own processor lending to a relatively newer concept of distributive-processor data routers. The three types of cards are control cards, line cards, and fabric cards. Generally speaking, a line card functions as an interface for the internal network of the router and the external network to which the router connects. Data packets entering and exiting the Terabit router typically do so through a line card. Fabric cards are a part of an internal data-routing fabric or network of the data router. Fabric cards act as individual routing points within the data router. Control cards are responsible for controlling both line and fabric cards in terms of their functions including initiating routing protocols, maintenance, special processing, control-message distribution, and so on.
In the prior art, most routers use a single processor for the processing of data packets. Some routers use more than one processor, however the number of processors is relatively small compared to a distributive data router as known to the inventor. When a data router of the prior art needs updating with new software such as newer versions of routing protocols, configuration software, or the like, the router being upgraded must be shut down and taken off line in order to perform the upgrade. After the upgrade is complete, the router must be re-booted to again join the network topology with its peers.
With efficiency of data routing in mind, it is clear that the loss of a router during upgrade procedures affects network performance through the particular portion of the network occupied by the router. Data must be re-routed through peer routers on to its destination, sometimes taking a much longer or even more constricted path or paths. It is desirable, however, that data packets be routed efficiently through a given network and connected routers at all times. Taking one or more routers down for upgrade can be a major inconvenience. The nature of a distributive router as known to the inventor provides processor architecture that may be treated differently than its single-processor counterparts where upgrading is concerned.
What is clearly needed is a method and apparatus for enabling a software upgrade to be applied to a distributive-processor data router without having to take the entire router off-line and without significant performance degradation during upgrade. A system such as this would enable, in most cases, automated upgrade of data routing protocol revisions, configuration changes and the like.
In a preferred embodiment of the present invention a method for performing an in-service software upgrade to a data router comprising steps of (a) providing a source node hosting an upgrade software package; (b) causing an upgrade command to be executed; (c) establishing, as a result of the upgrade command, a network session between the data router and the source node; (d) receiving the upgrade software at the router; (e) copying and distributing, within the router, the upgrade software to designated components slated for upgrade; (f) designating one or more components in the router as backups for each component to be upgraded; (g) backing up services and software running on each target element to be upgraded to designated backup element or elements while upgrade proceeds for the target element; (h) causing a switchover at the target element to the new software; and (i) repeating steps (g) and (h) until all the components slated for upgrade are upgraded.
In some embodiments the data-packet-network is the Internet network. Also in some embodiments, in step (a), the source location node is an FTP server and the upgrade software package is stored in memory of the server. In other cases the source location node may be a PCMCIA flash card installed in a network-connected computer, the upgrade software package residing on the card. In some embodiments the source location node may be a PCMCIA flash system residing within the router, the upgrade software package residing in the flash system.
In some cases execution of the upgrade command is manual and initiated by a user, while in other cases execution of the upgrade command is automated and triggered at a pre-set time. In some cases data network session is conducted over an Ethernet network between the source location node and the receiving component of the router. The location node may be an FTP server and the receiving router component may be a primary control card. In other cases the source location node is a personal computer. In some cases the data network session is conducted over the internet network between source location node and the receiving component for the router.
In some embodiments receiving the upgrade software comes after access and request by the router receiving component, while in other embodiments receiving the upgrade software comes after access and request sent to the router receiving component. Designated elements for upgrade may include control cards, line cards, or fabric cards.
In some cases the switchover command is automatic and executed after a certain period of time, while in other cases the switchover command is pre-configured to execute at a future designated time, the switchover managed by the primary receiving component of the router. In some cases notification of upgrade status is conducted in a separate and future data session. In some embodiments backup designations are made according to an algorithm for maximizing performance during the upgrade process.
In another aspect of the invention a system for upgrading a data router while the router is forwarding data packets over a data-packet-network is provided, comprising a node having access to the router by data link, a software upgrade package stored in memory residing in or connected to the node, a component residing in the router having access to the node over the data link, and a user station having access to the router for displaying upgrade status and results. The system is characterized in that at a specified time, the router component is caused to have access to the software upgrade package, receives and makes copies of image portions of the package, distributes the image portions to router elements within the router identified as targeted for upgrade, backs up functions of targeted elements to designated backup elements, and wherein the targeted elements each install their received image portion, reboot using the new image portion, and report back to the router component, which in turn reports status to the user station.
In some embodiments elements are upgraded in groups. Also in some embodiments backups are designated according to an algorithm for maximizing performance during the upgrade process. The data packet network may be the Internet network.
In some cases the node having access to the data router is an FTP server, and in some cases may be a personal computer. The data link may be an Ethernet network link.
In some cases the memory hosting the software upgrade package is PCMCIA flash memory, and in some the element residing in the router is a primary control card. In other cases the element residing in the router may be a line card. The user station can be a personal computer. In some embodiments the upgrade package includes image portions for control cards, line cards, and fabric cards.
In another aspect of the invention a router upgrade application for upgrading at least one targeted computerized element of a router is provided, comprising at least one runtime image, the image including parameters for element boot and operation once booted, an executable command for initiating the upgrade application including distribution and install, and an executable command for applying the at least one runtime image including boot directory reset and reboot instructions. The application is characterized in that the upgrade application functions automatically after a pre-determined or user selected time in cooperation with a primary router element to effect upgrade and reboot to all of the designated target elements of the router.
In some embodiments of the router upgrade application the computerized element comprises one of a control card, a line card, or a fabric card. In preferred embodiments the upgrade occurs while the router is forwarding data over a data-packet-network, and the data-packet-network may be the Internet network.
In some embodiments the initiation command is automatically executed according to a pre-determined time, and in others the apply command is automatically executed according to a pre-determined time.
There may be multiple backup router elements to assume the operational responsibilities of a target element being upgraded, and there may be a backup router element to assume the operational responsibilities of multiple target elements being upgraded simultaneously. In some cases targeted elements include cards of different types, and elements are upgraded in groups by type.
In embodiments of the invention described in enabling detail below, for the first time a system and method is provided wherein computerized machines, such as routers, may be upgraded while they continue to perform their on-line functions.